"The highlighted tokens are predominantly Vietnamese morphemes, syllables, or word stems, often at the beginning of words or as standalone syllables, including both lowercase and uppercase forms. These tokens frequently represent meaningful units in Vietnamese, such as prefixes, roots, or grammatical markers, and are often used to construct or modify words. The pattern reflects the tokenization of Vietnamese text into short, meaningful segments that align with the language's syllabic and morphological structure."
Score Type | Accuracy | Precision | Recall | F1 score | TPR | TNR | FPR | FNR |
---|---|---|---|---|---|---|---|---|
detection | 0.67 | 0.947 | 0.36 | 0.522 | 0.36 | 0.98 | 0.02 | 0.64 |
fuzz | 0.62 | 0.7 | 0.42 | 0.525 | 0.42 | 0.82 | 0.18 | 0.58 |